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Executive Summary 


This interim report presents preliminary data and observations from evaluations of Intel 
Teach to the Future being conducted around the world, and recommendations for 
building and refining this evaluation portfolio to ensure that findings will be instructive at 
the local, national and international level. The data presented here reflect the various 
evaluation processes that are underway, as well as those that are just beginning, in 
countries participating in Intel Teach to the Future. This interim report is intended to 
provide formative information for consideration by program staff as they develop and/or 
extend their evaluation plans, rather than summative information about overall program 
impact. The report is structured to present the following: 


¢ Preliminary data from the Core Surveys administered in four countries: Taiwan, 
Japan, India and the U.S. (n=11,986) 

¢ Themes derived from analysis of the evaluation reports submitted to EDC, site visits 
to four countries, conversations with education managers and evaluators, and EDC’s 
evaluation of the U.S. program. The report also presents programmatic and evaluation 
recommendations that address these themes. 

¢ Summaries of the evaluation plans and findings from twenty countries. 


The preliminary results from the Core Survey suggest that Intel Teach to the Future is 
having an impact on the teaching practice of participants. Not only did a large percentage 
of teachers report implementing a new technology-integrated lesson or activity since their 
participation in the program, but many also report that they are experimenting with a 
number of the project-based teaching strategies promoted in the training. These 
preliminary findings also suggest strong relationships between access to technical 
resources and rates of implementation, and an equally strong relationship between 
teachers’ perception of the relevance of the project-based teaching strategies presented in 
the training and rates of implementation. Some key findings from the Core Survey 
administered in four countries include: 


¢ 79% of teachers report implementing a new technology-integrated lesson or activity 
since the training. 

¢ Teachers who did not have access to computer labs were more likely to report not 
having implemented (39%) than those who had access (19%). 

¢ Teachers who had classroom computers were more likely to implement technology- 
integrated lessons than those who did not, and the more classroom computers teachers 
had, the more often they implemented. 

¢ The challenges to implementation most commonly cited by both teachers who 
implemented and teachers who did not were those related to lack of access to 
technology and lack of time. However, among those who had not implemented, lack 
of administrative and technical support were also cited frequently as obstacles, while 
those who had implemented did not cite these as frequently. 
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Teachers who agreed that the teaching strategies presented in the training were 
relevant to their teaching goals were more likely to implement technology-integrated 
lessons than those who did not. 

Nearly all teachers (94%) who implemented a technology-integrated lesson reported 
that their students were motivated and actively engaged in the lesson. 

More than 60% of teachers reported presenting lessons to students using technology, 
conducting research on the Internet, and accessing Internet resources for lesson 
planning more frequently since the training. 


The themes that we identified across countries were organized using a research-based 
framework for analyzing technology integration in educational environments. This 
framework describes four factors that facilitate or impede the integration of technology: 
infrastructure, professional development, administrative support and time. We also 
identified some key evaluation themes and issues that have emerged in multiple 
countries. In each category we make programmatic and evaluation recommendations for 
addressing these themes. 


Infrastructure is consistently raised as an issue by teachers working in a wide range 
of access conditions. We suggest making it explicit in the training that Intel Teach to 
the Future is designed to help teachers integrate project-based ICT curriculum within 
the given ICT environments in which they work, and focusing discussion in the 
training around the specific access conditions of participants. We also suggest 
collecting stories of innovative implementation in challenging access environments 
through case studies and contests. These would provide concrete examples for 
teachers who find it difficult to conceptualize how to integrate technology into their 
teaching. 

Teachers across the world have been overwhelmingly positive about the professional 
development they have received through Intel Teach to the Future. We suggest ways 
in which the program can build on this initial positive response. Some countries have 
already developed methods for providing ongoing professional development for their 
MTs through the use of MT groups who meet regularly to share ideas and 
experiences. These groups could be the focus of qualitative evaluations that explore 
the ways in which these groups allow MTs to learn from each other and provide 
follow-up support to teachers. Based on concerns expressed by teachers in a number 
of countries regarding their need for guidance in helping students make effective use 
of digital resources, we also suggest follow-up professional development in 
“information literacy” or “media literacy.” 

The relationship between administrative support and teachers’ ability to implement 
technology-integrated curriculum varies across national and local contexts. In order 
for the program to be effectively adapted to a range of environments, program staff 
need to be able to create programmatic structures that involve administrators at the 
level necessary for participants to receive the support they need to experiment with 
technology. Case studies of administrator workshops currently in place could shed 
light on how these initiatives can be integrated into programs in other countries. In 
addition, hierarchical, multi-level evaluations of educational systems can be designed 
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to provide information to program staff about the administrative and structural issues 
that their programs need to address. 

¢ Time constraints are consistently mentioned as an obstacle to technology integration 
by program participants. Although lack of time is a difficult problem to resolve, there 
are ways that program design can address the issue. At least one program has required 
that all participating schools have Master Teachers on site. Evaluation of this design 
element can enable program staff in other countries to understand whether this 
strategy allows participating teachers to more effectively and efficiently implement 
technology-integrated lessons. In addition, data suggest that offering incentives, 
especially in the early stages of the program, is essential for encouraging both MTs 
and PTs to dedicate their time to the training. Systematic exploration of the 
importance of incentives will better enable program staff to make decisions about 
how to allocate resources. 

¢ At this stage of our involvement with the international program, a number of 
evaluation issues have arisen. First, the core survey has already produced 
interesting, cross-country findings. However, we have identified some key areas of 
Core Survey development, administration and reporting that can be improved, such as 
the degree of standardization of the questions and communication about Core Survey 
requirements. On a broader level, we emphasize the need to establish realistic and 
meaningful criteria and indicators for program success. We also encourage the use of 
targeted case studies to explore various aspects of program design and a variety of 
educational contexts so that individual countries’ evaluations can inform program 
staff around the world. Looking to the future, we suggest creating standardized 
instruments for use in pre-service program evaluations and documentation of the 
knowledge base that exists among program staff. 


The interim report then presents summaries of the materials we have received from 
twenty evaluations being conducted around the world. Some countries with mature 
programs have submitted evaluation reports, the finding of which we summarize in this 
section. Other countries are at the preliminary stages of their evaluations, and for these 
we describe their evaluation plans. 
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Introduction 


Since March 2003, EDC’s Center for Children and Technology has been helping 
guide and coordinate evaluations being conducted in many of the countries taking part in 
the Intel Teach to the Future program. EDC’s goals for this international evaluation 
project are to: 


¢ Support the individual programs in establishing evaluations of Intel Teach to 
the Future in their countries. 

* Collect standardized survey data across countries. 

¢ Synthesize data across countries so that the international program staff can 
better understand how the program is working around the world. 


As part of this work, EDC has created the following resources: 


¢ Guidelines for evaluation. 

¢ A RFP template for education managers to use to solicit evaluation proposals. 

¢ A Core Survey to be administered worldwide. 

¢ A data entry form for recording and delivering Core Survey data. 

¢ A list of deliverables that identifies all of the information and materials 
country programs need to send to EDC. 


Along with developing these materials, EDC has maintained regular communication 
with program staff around the world, consulting programs on their evaluation plans and 
providing feedback on materials they develop and on proposals they receive from 
prospective evaluators. EDC has received and reviewed the evaluation reports produced 
by these programs once they are completed. In addition, EDC has continued to collect 
and report on the end-of-training survey data from numerous countries. Finally, EDC 
evaluators have made visits to Russia, Mexico, India and China to learn more about these 
programs from Intel Teach to the Future staff, educators, students and others involved 
with the program and to discuss evaluation goals, plans, and findings in more detail. 

This Interim Report presents reviews of survey data collected to date and preliminary 
observations from a number of sources—the Core Survey, the country-specific reports 
submitted by individual programs, conversations with program staff and evaluators 
worldwide, the information gathered from our site visits and our evaluation of the U.S. 
Intel Teach to the Future program. 

In order to make sense of such a diverse array of data, we have used a research-based 
conceptual framework to structure our analysis. Several factors typically facilitate or 
impede the implementation of technology-based applications in K-12 educational 
environments (Mandinach & Cline, 1994). These factors have proven to be pervasive 
across a broad range of applications, innovations, and programs. The processes by which 
Intel Teach to the Future is being implemented, and the obstacles and opportunities 
associated with them, will be examined through the framework of these factors. 
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Infrastructure. Technology-based programs, such as Intel Teach to the Future, 
cannot be implemented without the necessary technological infrastructure. There are two 
interrelated components to infrastructure. First there is the need to have enough hardware 
and software to meet the pedagogical objectives of the schools. More often than not, 
schools must make do with less than optimal numbers of computers or with setups that do 
not correspond to the school’s stated curricular goals, but regardless there must be a 
minimal level of computer access. Some schools prefer the laboratory model, while 
others prefer to distribute the computers across individual classrooms. There is no right or 
wrong model, only what matches the specific needs of the school and its instructional 
goals and that provides the best possible physical access to the hardware. The second 
infrastructure issue is connectivity or access to the Internet. Programs such as Intel Teach 
to the Future and the larger Innovation in Education initiatives require connectivity, in 
part to provide access to web-based applications such as Seeing Reason. More generally, 
the principles that underlie the program and the project-based learning it seeks to support 
necessitate Internet access. Without access, Intel Teach to the Future will have limited 
impact. There are many levels of quality and reliability of access to the Internet that 
teachers may have. However, the type of robust, high-speed access familiar to 
professionals working in office settings is still quite rare in classrooms. For example, 
teachers who nominally do have Internet access may have to share a single phone line 
with the building administrator. This level of access is not adequate to their instructional 
needs. 

Professional development. Intel Teach to the Future addresses teacher professional 
development through the introduction of technology-based applications to support 
project-based learning. The implementation of technology requires that teachers, both at 
the pre-service and in-service levels, acquire new technical knowledge and skills as well 
as reflect on and adapt their instructional practices. Effective technology professional 
development must help teachers to identify and meet specific educational needs and 
should be ongoing, rather than generic and short-term. One-shot training frequently 
leaves teachers frustrated because it does not provide them with the chance to think 
through, explore, and reinforce the concepts being covered. Equally important in 
professional development is for training to address the paradigm shifts and pedagogical 
changes that are often associated with the use of many technology-based applications. In 
Intel Teach to the Future, as in some other high-quality professional development 
programs, the pedagogical emphasis is on constructivism and the creation of active, 
student-centered learning environments. Training must prepare teachers to deal with the 
fundamental philosophical and practical changes that emerge when shifting from the 
“sage on the stage” or the imparter of facts and information to the “guide on the side” or 
facilitator/coach in which teachers assist students to actively construct knowledge and 
engage in learning processes. 

As Papert (1997) and Salomon & Almog (1998) have noted, the more creative and 
broad-based the particular application, the greater potential for paradigm shifts and 
discernable outcomes. However, the greater the paradigm shift, the more difficult the 
implementation and transition from previous instructional methods. By contrast, though 
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more easily integrated applications often show less dramatic impact, they often get 
implemented more broadly. Salomon & Almog (1998) state: 


The more a technology, and it usages, fits the prevailing educational philosophy 
and its pedagogical application, the more it is welcome and embraced, but the less 
of an effect it has (p. 224). 


Intel Teach to the Future’s strength lies in its broad applicability. From our 
perspective, the program does not take direct aim at paradigm shifts in practice, but rather 
finds appropriate niches in the curriculum to introduce project-based learning and 
technology-based instructional activities.. 

Administrative support. The implementation of any new professional development 
program, not just one involving technology, requires a certain level of buy-in from 
various levels of administration. Principals or headmasters can make or break any 
program. They can either facilitate or impede implementation. It is always optimal to 
have not only the approval but also the active support and engagement of the 
administrator in any program that seeks to cause real change in teachers’ practices. Lack 
of support from administrators is nearly always problematic. In some instances, however, 
innovative programs can be effective despite a lack of administrative support. A grass 
roots or bottom-up model led by the teachers can succeed under certain circumstances, 
even when there is no support from the administration, but only if teachers have 
significant flexibility in implementing the curriculum, and can easily access technology 
on their own. However, the significance of administrative support in technology 
implementation is closely tied to the political and cultural realities of the particular 
educational system. In countries, states/provinces, regions, educational systems, and 
schools that have more tightly hierarchical structures, administrative support may be the 
key factor in determining whether technology can become a part of the teaching and 
learning process. In areas with a more decentralized educational system, where teachers 
have greater control over what they do with their students, administrative support may be 
not be as crucial. 

Time. Teachers need time to be trained, acquire necessary knowledge and skills, 
assimilate new concepts, and practice with new technology. They must have sufficient 
time and resources to develop new curricula and instructional strategies, and then to 
integrate them into classroom practice. Such integration and implementation requires 
patience, skills, and the ability to perceive cues about what works effectively or does not 
and then use such feedback to further improve pedagogical practices. 


Taken together, these preceding factors—infrastructure, professional development, 
administrative commitment, and time—form an interacting set of variables that play a 
significant role in determining whether ICT can become a significant learning tool in 
complex systems called schools. Intel Teach to the Future directly addresses one factor: 
professional development. The governments and educational systems in the participating 
countries are primarily responsible for addressing the other three factors: infrastructure, 
administrative support and time. Regardless of where responsibility lies, the evaluations 
of Intel Teach to the Future can be designed to explore how all four factors influence the 
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effectiveness of the program. By understanding their influence, Intel staff can make 
informed decisions as they adapt the program to meet the needs of the educators and 
schools their programs serve. 


This report is divided into four sections. First, we explore findings from the Core 
Survey. Second, we examine themes that cross the countries. Third, we provide brief 
evaluation summaries from the twenty countries that have sent us information. Finally, 
we draw conclusions and discuss future directions. 
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Core Survey Results 


In this first analysis of the Core Survey data, we look at the findings from surveys 
conducted in Japan, India, the U.S. and Taiwan. The Philippines has also completed the 
Core Survey, but the data are not currently in a format that will allow us to integrate them 
with the other data. We expect to integrate these data soon. In addition, China just 
completed its survey, and the evaluators are reviewing and analyzing it now. Brazil sent 
their Core Survey data on December 18. We will be able to provide an updated analysis 
of the Core Survey data that includes these three additional countries in January. 

Table 1 provides information about the percentages of respondents from each 
country. We conducted the analysis so that each teacher’s voice is weighted equally, 
rather than looking at results on a country-by-country basis. However, because the 
number of respondents varies a great deal from country to country, certain countries have 
greater representation in the total percentages than others. 


Table 1. Countries represented in Core Survey findings 


Country Frequency Percent 
1. Japan 583 4.9 
2. India 1848 15.4 
3. USA 4277 35.7 
4. Taiwan 5278 44.0 
Total 11986 100 


Intel Teach to the Future materials 

In the Core Survey, we asked teachers to tell us how often they made use of the 
materials provided in the Intel Teach to the Future Program after they had completed the 
program. As the Figure | below indicates, teachers made use of the manual about as often 
as they made use of the CD-ROM. For both resources, roughly 35% of the respondents 
report using the materials 4-10 or more than 10 times since their training. The largest 
groups (43.8% for the manual and 41% for the CD-ROM) have only used these resources 
1-3 times since the training. One-fifth of respondents said that they had not used these 
resources at all. 
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Figure 1. Use of Intel Teach to the Future Materials 


Implementation of technology lessons or activities 
A primary goal of Intel Teach to the Future is to have participants bring the 


knowledge and skills they gain through the training back to their classrooms. In the Core 
Survey we ask teachers to report on whether they have introduced a new technology-rich 
lesson or activity into their classroom teaching. The Core Survey analysis indicated 
that many teachers were implementing new technology lessons and activities (see 
Figure 2). Only 21% stated that they had not used computer technology at all with their 
students. About 30% of teachers reported having their students use computer technology 
in learning activities more than once a month; and roughly the same number had students 
use technology in their lessons less than once a month. 


EDC/CCT Intel Teach International Interim Report 
96 Morton St. NY, NY 10014 Page-9 


35 4 


30 4 


Percentage 


More than once a_ About once a Less than once’ Have not used 
month month a month 


Figure 2. Implementation of technology lesson or activity 


Relationship between access and implementation 
Because access conditions vary dramatically, not only from country to country but 


also from school to school, it is difficult to define implementation success for every 
context. In cases where resources are limited, even implementing less than once a month 
would indicate a substantial change in some teachers’ practices. For this reason, it is 
important to examine responses about implementation in relation to the kind of access 
conditions that teachers experience. 

The Core Survey asked teacher to provide information about the kind of technology 
access they have in their schools. A majority of teachers (77%) have access to a computer 
lab or media center. Within that group nearly all teachers had Internet access in their 
computer labs (93%). We found, not surprisingly, that access to computer labs was a 
key factor in whether or not teachers were able to implement a new technology 
lesson or activity. As Figure 3 illustrates, teachers who reported that they did not have 
access to a computer lab were more likely to say that they had not implemented a 
technology lesson (39%) than those who did have access (19%), and those who had 
access to computer labs were also more likely to implement more than once a month 
(30%) than those who did not (16%). 
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Figure 3. Frequency of implementation by access to computer lab 


Along with asking teachers to report on their access to computer labs, we asked them 
about their access to classroom computers. Forty-one percent of respondents had no 
computers in their classrooms, 29% had only one computer in their classrooms, 16% had 
2-4 computers, 5% had 5-7 computers and 8% had more than 7 computers in their 
classrooms. Of those who did have classroom computers, 78% had Internet access for all 
of them;18% had Internet access for some; and only 4% had no Internet access for their 
classroom computers. There was a strong relationship between access to classroom 
computers and the frequency with which teachers implemented technology lessons. 
As Figure 4 shows, 50% of teachers with 7 or more computers in their classrooms 
implemented technology lessons more than once a month, while only 27% of teachers 
with no classroom computers were able to implement that often. Conversely, even those 
teachers who had just one classroom computer were half as likely to say they had not 
implemented (17%) as those who had no classroom computers (33%). Very few teachers 
with 2 or more computers in the classroom reported not having implemented at all (16%). 
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Figure 4. Frequency of implementation by classroom computer 


Implementation challenges 
Teachers face many challenges in their teaching that can make it difficult for them to 


implement technology. The Core Survey asked those teachers who had not implemented 
a lesson to give feedback on the reasons why they were not able to do so. Figure 5 shows 
what those teachers who had not implemented any new technology lesson cited as the key 
obstacles to technology integration. The findings on implementation obstacles are in 
alignment with much of what we suggested in the introduction, namely that 
infrastructure, time and administrative support are some of the most influential 
factors in determining whether teachers can actually use technology in their 
teaching. Respondents cited the following as the greatest obstacles to implementation: 
necessary computers were not available (64% agree or strongly agree), necessary 
software was not available (56% agree or strongly agree), lack of adequate planning time 
(58% agree or strongly agree), lack of administrative support (50% agree or strongly 
agree) and lack of technical/instructional support (51% agree or strongly agree). It should 
be noted that the items cited least often as reasons for not implementing include that 
respondents did not consider implementing the lesson (14%) and that it would not help 
them meet the learning standards they are required to meet (15%). These findings are 
important, because they indicate not only that teachers are willing to implement 
their lessons, but that they also feel these lessons meet their teaching needs. 
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Figure 5. Reasons for not implementing technology lesson or activity 


Those teachers who were able to implement a technology-integrated lesson or activity 
faced challenges as well. The Core Survey asked those teachers who had implemented a 
technology-integrated lesson to tell us about the challenges they faced in doing so (Figure 
6). The challenges they cited resemble those identified by the teachers who did not 
implement. Access-related obstacles, such as not having enough computers available, 
were significant for both groups (53% of implementers strongly agree or agree that this 
was an obstacle, compared to 64% of non-implementers). Various items related to time 
constraints were also cited as problems by both sets of teachers. Sixty-four percent of 
respondents who had implemented agreed or strongly agreed that time constraints 
prevented them from completing the entire lessons, and 51% agreed or strongly agreed 
that it was difficult to schedule adequate time in their computer labs. Interestingly, two 
key challenges that non-implementers cited more often than implementers as significant 
obstacles were administrative and technical/pedagogical support. Fifty percent of non- 
implementers said they did not receive adequate administrative support, while only 29% 
of implementers cited lack of administrative support as an obstacle. Likewise, although 
51% of non-implementers said they did not receive adequate technical/instructional 
support, only 32% of implementers saw this as an obstacle. These findings suggest that 
administrative and technical/instructional support may be somewhat more 
influential in determining whether teachers are actually able to implement 
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technology-integrated lessons in their teaching. Access to computers is certainly an 
important factor but perhaps one that can sometimes be overcome. Both groups 
considered lack of time a major obstacle or challenge, but may not necessarily be one that 
keeps them from attempting to implement new technology-rich projects. 
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Figure 6. Obstacles faced by teachers who implemented technology lessons 


Teachers’ perceptions of student response to technology-integrated lessons 
We also asked those teachers who did implement a new technology-integrated lesson 


or activity about student responses to these lessons. As Figure 7 demonstrates, across all 
of these questions, teachers expressed positive opinions. Respondents agreed or strongly 
agreed that when implementing these lessons: 


Students were more motivated and actively involved in the lesson (94%) 
Students worked together more often (85%). 

Students helped one another with the technology (86%). 

Student work showed more in-depth understanding than prior, comparable. 
assignments (75%). 

Student work was more creative that previous, comparable assignments (78%). 
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Figure 7. Student responses to the technology lessons or activities 


Teacher response to pedagogical strategies presented in training 
The Intel Teach to the Future training focuses on both helping teachers with 


technology integration and introducing participants to project-based pedagogy. We asked 
teachers about these elements of the program. The majority of teachers (62%) stated that 
it was “very true” that the program helped them understand how to integrate technology 
into their teaching, and 36% felt this was somewhat true. Slightly more than half (53%) 
of the respondents said that it was “somewhat true” that the teaching strategies presented 
in the training were new to them. Only 12% indicated that it was “not true at all” that the 
ideas were new. Slightly more than half (53%) of the respondents felt that it was “very 
true” that these teaching strategies were relevant to their teaching goals and 44% reported 
that this was “somewhat true.” Only 3% felt that this was “not true at all.” 

We analyzed the relationship between respondents’ perception of the relevance of the 
teaching strategies presented in the training to their teaching goals and how often they 
implemented their technology-integrated lesson. We found a very positive association 
between the degree of relevance and the frequency of implementation. As Figure 8 
illustrates, those respondents who felt that the teaching strategies were relevant to their 
teaching implemented more often than those who felt they were not relevant. Almost half 
of the teachers who said that the teaching strategies were not relevant did not implement 
any technology project. More than a third of respondents who said the teaching strategies 
were relevant implemented more than once a month. This finding is significant for 
understanding what factors motivate teachers to introduce technology into their 
teaching. 
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Figure 8. Frequency of implementation by relevance of teaching strategies 


In order to better understand whether the program was having an impact on 
pedagogy, we asked teachers whether they engaged in certain teaching practices more 
often, less often, or the same amount after they had completed the training. The 
responses given by teachers to this question about change in teaching practice 
suggest that teachers are moving through the initial stages of technology use 
common among educators. Often teachers must first integrate technology into their 
familiar forms of teaching before they can begin attempting more innovative pedagogy. 
Figure 9 shows that these teachers were presenting to their students using technology 
(61%), conducting research on the Internet (63%), and accessing Internet resources for 
lesson planning (63%) more often since the training. Slightly fewer reported engaging 
more often in those forms of pedagogy more closely associated with project-based 
learning, such as using rubrics for evaluating student work (47%) and using essential 
questions to structure lessons (51%). It is notable that the only teaching practice that a 
considerable number (24%) of teachers indicate they do less often after the training is use 
a textbook as a primary guide for instruction. 
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Figure 9. Change in teaching practices 


We also wanted to understand what kind of learning opportunities teachers were 
offering students after they had completed the training (Figure 10). On most of the items, 
teacher responses were closely split between “no change” and “do this more often,” 
which is quite significant given the difficulty many teachers have in changing their 
practice, especially in such a short span of time. The items for which more teachers said 
they “do this more often” than “no change” include having students work on computers 
to do lessons or activities during their class time, asking students to use computers in 
non-school settings (home, library, Internet café), having students engage in independent 
research on the Internet, and having students present to the class. As with the previous 
question, for the most part, these items represent the more familiar forms of teaching than 
some of the other items, such as having students review and revise their own work, and 
allowing them to choose their own research topics. Again, these findings suggest that 
teachers are moving through the initial stages of growing comfortable with 
technology use. It should be noted, though, that a fairly large number of teachers (from 
32-44%) do report doing the less traditional activities more often since the training. 
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Figure 10. Change in student activities 


Summary of Core Survey findings 
The overall findings from this initial analysis of Core Survey data from four countries 


suggests that Intel Teach to the Future is having a notable impact on teachers’ use of ICT 
in their classrooms. The program is encouraging a substantial number of teachers to 
experiment with new pedagogical strategies. When asked to respond to specific items 
related to the kinds of teaching and learning taking place in their classrooms since the 
training, for the most part, teachers responded very positively. For example, a majority of 
teachers reported that students were engaged in and motivated by the ICT projects, 
students worked together and helped one another, students’ work showed greater 
creativity and in-depth understanding and students’ different learning styles were 
addressed by these projects. On questions about how they are using ICT to support their 
instructional practices and the activities and opportunities they offer their students, the 
survey findings indicate that majorities of teachers are moving through the initial phases 
of technology integration by using ICT to support more familiar forms of teaching, and 
relatively large minorities are experimenting with more substantial forms of project-based 
pedagogy. 

The data are also consistent with our conceptual framework, presented in the 
introduction, which identifies the key factors that teachers report as challenges or 
obstacles to implementing new ICT projects, such as inadequate infrastructure, lack of 
administrative or other kinds support (such as technical and instructional) and time 
constraints. There was a clear relationship between teachers’ access to technology, both 
computer labs and classroom computers, and the frequency of implementation. There was 
also an equally clear, and perhaps even more interesting, relationship between the 
frequency of implementation and whether respondents felt the teaching methods 
presented in the training were relevant to their teaching goals. These findings provide a 
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useful overview of teachers’ experiences with and responses to the program. Additional 
evaluations, particularly those that make use of qualitative methods, such as focus groups 


and case studies, will provide more in-depth data that will allow us to better understand 
many of the issues raised here. 
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Themes Identified across Countries 


As we noted in the introduction, the research in educational technology has indicated 
that there are four main factors that facilitate effective technology integration in 
classrooms. These factors are infrastructure, professional development, administrative 
support and time. Because the key goal of the Intel Teach to the Future program is the 
improvement of education through the use of project-based ICT in the classroom, we will 
use these factors to provide a conceptual framework for our findings from evaluation 
activities to date. Following are some of the major themes we have derived through our 
analysis of data from a range of sources: the U.S. program evaluation, the evaluation 
reports submitted by the other countries participating in the program, the Core Survey 
data, our conversations with program staff and external evaluators, and our site visits to 
Russia, Mexico, India and China. As we explore these themes, we also include 
preliminary recommendations for addressing issues related to these themes. Some of 
these recommendations are programmatic, but many are for evaluation activities that 
could be used to explore these themes more extensively. 


Infrastructure 

Having an adequate technological infrastructure is essential for teachers to be able to 
integrate ICT into their teaching. However, there is no definitive definition of what 
constitutes “adequate” technical infrastructure. The countries, regions and schools that 
Intel Teach to the Future reaches are incredibly diverse, with a wide range of ICT 
capacities. In some cases, teachers in schools that have what might be considered 
inadequate ICT resources are able to find creative ways of implementing technology- 
enhanced projects. Likewise, teachers in schools that would, by most standards, be 
considered technology-rich environments may have difficulty implementing technology- 
integrated lessons for a host of other reasons. 

We recognize that Intel Teach to the Future does not have the resources to build up 
the technical infrastructures of all of the schools it serves worldwide. However, issues of 
infrastructure can be addressed by the program and the evaluation of the program, so that 
limited access to technology does not have to become an insurmountable obstacle and so 
successful strategies for circumventing infrastructure limitations can be shared with the 
Intel Teach to the Future community and the larger educational community as a whole. 


Explicit guidance in coping with resource limitations 

A review of the evaluations that have been conducted thus far of the Intel Teach to 
the Future program indicates that nearly all teachers are concerned about resource and 
Internet access limitations, from teachers in South African townships who have to bring 
their students to a community center to use computers to U.S. teachers with multiple 
computers in their classrooms. One of the implicit goals of Intel Teach to the Future is to 
empower teachers to see themselves as capable technology users. One way to create this 
sense of empowerment is help teachers understand how they can work successfully 
within the technological constraints they encounter. 

Teachers represented in the various evaluations that have been conducted, especially 
in the programs that are in the early stages, have expressed concern that they will be 
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unable to make use of the skills and knowledge they may gain from the training because 
of their limited technical resources. Initial communication about the program and the 
communication that occurs during trainings may not be explicit enough in articulating 
that Intel Teach to the Future is designed to help teachers integrate project-based ICT 
curriculum within the given ICT environments in which they work. Although the training 
currently touches on infrastructure conditions of the participants, it could more directly 
acknowledge the specific kinds of access participants have and provide examples of how 
teachers around the world have learned how to deal with a wide array of situations. 

One consistent issue that program staff in the countries we have visited have raised is 
the issue of the digital divide, particularly as it pertains to urban and rural areas. In 
countries such as Russia, Mexico and China, the inequalities in infrastructure between 
urban schools and rural schools are extreme, and can seem on the surface to create 
obstacles impossible for teachers to overcome. These countries and others with similar 
challenges could use their evaluations to focus on schools and regions with particularly 
challenging infrastructure conditions. By designing targeted case studies that explore how 
Intel Teach to the Future participants deal with the limited ICT environments in which 
many of them work, these studies could provide useful information that would allow 
program staff worldwide to better understand the needs of participants and construct 
realistic expectations among themselves, Master Teachers (MTs) and Participating 
Teachers (PTs) of what the program can achieve. 


Showcasing/investigating creative implementations 

Many of the Intel Teach to the Future programs around the world, and the overall 
Intel Teach to the Future program, have made a point of showcasing teachers who create 
innovative, project-based ICT lessons and activities. By highlighting examples on the 
Intel Teach to the Future website, and offering awards to teachers in some countries, the 
program has sought to reward teacher innovation. However, there are many ways to 
conceptualize innovation and creativity. For teachers working under challenging 
infrastructure conditions, simply implementing their unit plans once a year is an 
impressive achievement. In order to explicitly address and acknowledge the infrastructure 
conditions teachers experience, the worldwide program and the specific country 
programs could begin to recognize creative, innovative implementation solutions as well 
as innovative projects in their award programs and on the Intel Teach to the Future web 
site. These examples would focus on how teachers learn to cope with challenges in order 
to bring their new skills and knowledge into their classrooms. By collecting these stories 
for the web site and through the awards programs, program staff, senior trainers and 
Master Teachers would have a resource to draw upon for helping new participants 
conceptualize implementation and see what it looks like in a diverse array of classrooms. 


Professional Development 
As we stated in the introduction, the main factor in successful ICT integration that 


Intel Teach to the Future addresses is, obviously, professional development. Through 
nearly every means by which we have been able to analyze the program (surveys, site 
visits, the U.S. evaluations, evaluation reports from the different countries) teachers have 
been overwhelmingly positive about the training they receive through this program. They 
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are equally positive about their experiences in the classrooms when they begin to 
implement the project-based pedagogy and ICT curricula they learn about and develop in 
the training. The themes we explore below are ways in which programs either have or 
could supplement the basic professional development to potentially enhance the 
program’s impact. 


Master Teacher follow-up and support 

Master Teachers are the key to sustainable impact for Intel Teach to the Future. MTs 
throughout the world serve as the conduits for communication between program staff and 
participant teachers, and are the best resource for identifying the needs of teachers and 
tailoring program activities to support them in their efforts. For this reason, it is likely 
that programs that include strategies for keeping MTs involved and invested in Intel 
Teach to the Future will meet with greater success. Programs in some countries, such as 
India, are making deliberate efforts to provide ongoing professional development and 
support to their MTs. The data from surveys and interviews with MTs indicate that they 
feel that one of the best things about the Intel Teach to the Future is that it gives them an 
opportunity to collaborate with peers. Creating ongoing opportunities for MTs to meet 
with and learn from each other is likely to be welcomed. In addition, because MTs are the 
people who will be in charge of delivering further professional development to 
participant teachers, their skills must be updated on a regular basis. Because many 
country programs do not have any systematic plans for MT follow-up and support, 
documenting the activities of the MTs groups in India and tracking the kinds of support 
MTs can then offer teachers as a result would make the benefits of including MT follow- 
up more evident for other education managers as they develop the programs in their own 
countries. 


Guiding students in analyzing, synthesizing, and presenting information effectively 

Feedback from teachers in various countries has suggested that teachers need 
additional guidance in helping students make effective use of information resources. 
Although teachers observe that students are very engaged in project-based ICT lessons, 
they are concerned that students are not always capable of analyzing and synthesizing the 
information they access or of presenting their work in ways that show that they really 
understand the material. Many teachers are not accustomed to having their students 
working with a wide variety of information resources or with evaluating student ICT 
products, so introducing students to the idea of “media literacy” or “information literacy” 
may not be in their instructional repertoire. However, there has been a good deal of 
research and curriculum development in this area that could be used in follow-up 
trainings for MTs and PTs, or which at least could be made available on the Intel Teach 
to the Future web site for teachers to access. 


Administrative support 
One factor in the ICT implementation equation that the Intel Teach to the Future 


program has not addressed fully is administrative support. This may be due in part to the 
fact that the program was developed in the U.S. Although administrative support is 
certainly a significant factor in the success of ICT implementation in U.S. schools (much 
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of the research on which we base this conceptual framework was conducted in the U.S., 
and EDC’s own research has focused on the importance of administrative support), U.S. 
teachers, relatively speaking, may have greater flexibility in integrating innovative 
projects than teachers in other countries (although this may be less true today than it has 
been over the last decade). In addition, greater access to technology in the U.S. and other 
developed countries may make administrative support less crucial, since teachers with 
classroom computers or regular access to computer labs may not be as reliant on 
administrators to give them opportunities to implement their projects. However, even in 
the U.S. evaluation of Intel Teach to the Future we found that administrative support 
contributed to the sustainability and institutionalization of program goals. 

Because Intel Teach to the Future is committed to adapting its program to meet the 
local needs of all of the countries it serves, it is essential for program design and 
evaluation to reflect the relative importance of administrative support in the success of 
ICT implementation. Some programs (such as those in Pakistan and India) have already 
established mechanisms that address administrative support. Below we describe how 
these developments can be investigated and built upon. 


Administrator workshops 

In our visits to different countries, especially in China and India, we observed that the 
program was having a notable impact in those schools that had strong leadership and 
support from the principal. Administrators in these schools helped establish for teachers 
the conditions that could allow for sustained impact, such as designating time in the 
school day for teachers to plan for technology integration. The India program has started 
mandating that administrators in schools participating in Intel Teach to the Future attend 
workshops about the program. Case studies focused on these workshops and on the 
ensuing activities of administrators who participate could prove a rich resource for 
information about how to educate administrators so that they are invested in the program 
and in the effective use of ICT for teaching and learning. 


Enhancement of administrator workshops 

Evaluations conducted by EDC of other technology professional development 
programs have found that administrators have very different needs from teachers when it 
comes to technology professional development. Just as the Intel Teach to the Future 
program addresses issues of significance to teachers in order to make the training relevant 
and useful, administrator professional development must also directly address issues that 
are important and relevant to them. It may not be particularly useful for administrators to 
participate in the same training in which teachers participate. Rather, administrator 
workshops could focus on what good technology integration looks like, how to use 
technology tools to accomplish administrative goals, such as developing technology 
plans, accessing information for school improvement, and creating effective presentations 
for staff meetings or for reporting to education officials. 


Hierarchal evaluation 
Schools are dynamic and complex organizations made up of interrelated components 
and levels. It is essential to take into consideration the interactions among these 
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components. Administrative support requires that many actors at different levels of the 
educational community work together to build supportive environments for teachers. In 
order to examine the complexities of this issue, an evaluation can take a “systems 
approach.” This means that evaluators must look into multiple levels of the education 
system to understand how certain decisions have an impact on local administrative and 
teaching practice, and conversely how local teaching and administrative practices may 
influence higher-level decision-making. For example, in India changes are being made to 
the national exams that will make it more desirable for teachers to use project-based 
learning in their teaching. This may be one of the reasons why many schools have 
embraced Intel Teach to the Future. By undertaking a multi-level evaluation, programs 
may be better able to understand the local, regional and national context in which their 
program is situated, how aspects of that context affect the program, and how the program 
may be helping to shape the context. 


Time 

Not only is time identified in the research literature as a significant factor in the 
successful integration of ICT in education, but data from surveys, interviews and focus 
groups conducted in evaluations of this program indicate that time constraints present 
significant obstacles to Intel Teach to the Future participants. On the surface, time 
constraints may appear to be an obstacle that a program such as Intel Teach to the Future 
cannot be expected to overcome. Obviously, Intel Teach to the Future cannot generate 
more time in the day for teachers. However, the evaluation conducted of the program 
reveals that there are, in fact, ways to address the issue of time in the program design, and 
additional evaluation could shed even more light on ways in which programs in different 
countries are attempting to deal with time constraints. 


School-based Master Teachers 

As we learned in the U.S. evaluation, MTs can be an invaluable resource for 
participant teachers working in the same school. Often, the reason teachers are unable to 
find the time to implement project-based ICT is because they do not have the necessary 
support for planning and for dealing with technical problems. Administrative support can 
help alleviate time pressure if principals designate time in the school day or week for 
teachers to plan their ICT-enhanced lessons, but having the support of school-based MTs 
can also help teachers utilize time more effectively by providing technical and 
pedagogical assistance when they need it. The India program requires that all schools that 
participate in the program have school-based MTs. A focused case study on the ways in 
which these MTs assist their peers and help create supportive communities by, for 
example, offering ongoing training or collaborative groups, could provide information 
about the effectiveness of this program design. 


Incentives 

Not only do teachers find it difficult to have time to implement their Intel Teach to 
the Future units, but they also have difficulty finding time to participate in the program. 
The time commitment required of MTs is even more onerous than the significant 
commitment required for PTs. Data from evaluations of this program suggest that 
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incentives for PTs and MTs are essential to the success of the program, especially when 
the program is just beginning. PTs and MTs in countries such as the U.S., Russia and 
South Africa have observed that the training is very time consuming, and there is little 
reason why they would be involved with the program without specific incentives for 
them. Relying on teachers’ interest in technology and improving their skills is not 
enough. In some cases, it is difficult for teachers to attend trainings because of 
transportation problems and family commitments. In mature programs, MTs and PTs 
come to see the value of the training, the methods and the use of technology, but this is 
not self-evident from the start. Once aspects of the program become institutionalized 
within schools, incentives may be less necessary, but this will not happen unless there is a 
critical mass of MTs and PTs who complete the program and are committed to its goals. 


Evaluation issues 

The themes explored above are related to the theoretical framework based on the research 
literature. Below, we look at some specific issues related to the evaluations that are taking 
place or may take place worldwide and offer some recommendations specific to these 
evaluations. 


Core survey 

A standardized survey is an important mechanism for gathering consistent data from 
Intel Teach to the Future participants all over the world. However, there are some 
technical challenges related to the Core Survey that will need to be resolved. After initial 
delivery of these data, three key issues have come to light: 

1. It has become apparent that keeping the Core Survey questions as intact as 
possible, and only changing the order when absolutely necessary, is crucial to 
efficient and successful reporting and analysis. For example, there are certain 
questions that are related to other questions and ask for respondents to skip 
questions based on their responses. The data will not be comparable if 
respondents are not given the same instructions for skipping to different 
questions, so these questions can not be moved to different areas of the survey. 

2. Some of the evaluators and education managers are not aware of the Core Survey 
data entry form. This document was part of the overall Core Survey package, 
along with the formatted and unformatted Core Survey and the Coding Key, but 
apparently some program staff either did not receive all of these materials or were 
not adequately informed of their purpose. Communication about the materials and 
processes required for the worldwide evaluation is in need of improvement. We 
will use the Education Summit as an opportunity to clarify the materials and 
processes related to the Core Survey administration, data collection and reporting. 

3. The process of designing and distributing the survey, providing feedback and 
receiving and analyzing the data has made it clear that it would be most efficient 
for us to be in direct contact with the evaluators when discussing technical aspects 
of the evaluation and the instrumentation. Although we realize that language 
issues may make this difficult, whenever possible, it would be beneficial for us to 
communicate directly with evaluators so that the process of instrument 
development, analysis and reporting can be streamlined. 
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Case studies 

After reviewing a number of the evaluation plans set forth by different evaluation 
agencies, we observed that when the plan calls for qualitative methods, the selection of 
case study schools is usually focused on finding schools that are representative 
(demographically, geographically, socio-economically) of the whole population of 
participating schools. This is not a problem in itself, but representativeness is not the only 
valid criteria for selecting schools for case studies. We need to encourage program staff 
and evaluators to be creative about school or district/region selection and consider 
sampling strategies that are most appropriate to the questions they seek to answer. 
Because of limited resources, the case studies should be designed to collect the 
information that is most relevant for learning about and improving the program. Program 
staff need to think about the key goals of their particular program and investigate those 
cases that are best suited to provided them with data about whether those goals are being 
achieved. For example, if one of the main goals of Intel Teach to the Future in a certain 
country is to help provide more girls with the opportunity to make use of technology in 
education, then case studies could focus specifically on what is happening in girls’ 
schools that have a significant number of Intel Teach to the Future trained teachers. If one 
of the goals of the program is to help close the digital divide between students in urban 
and rural areas, contrasting case studies could be conducted at a rural school with very 
limited access to technology and an urban school with greater access. These kinds of 
cases may not be representative of the schools in general that are being reached, but they 
may be important to target for case studies because they can highlight issues that might 
not arise at other schools. 


Pre-service survey 

Program staff and evaluators from a number of countries have expressed the desire to 
have a standardized pre-service survey, much like the end-of-training and Core Surveys. 
Currently there is no systematic way to look at the pre-service program, nor is the pre- 
service program defined uniformly across all countries. Education managers and 
evaluators would appreciate receiving instruments for use in evaluating the pre-service 
component. The development of such as instrument would rely heavily on the 
involvement of evaluators, program staff and perhaps faculty in programs around the 
world, because the existing instrumentation developed for looking at pre-service in the 
USS. is in need of refinement and, regardless, would not be appropriate for many of the 
other pre-service programs. 


Documentation of program designs 

Short of direct local observation, probably the richest source of information about the 
Intel Teach to the Future program comes from the program staff. Staff members have 
first-hand knowledge of the kinds of challenges the program faces in a variety of 
environments, and techniques that have been used to solve those problems. Although 
there is copious documentation in various forms about the program, we are not aware of 
any systematic process for capturing the knowledge that program staff have acquired and 
using it for program improvement, staff professional development and contributing to the 
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knowledge base. We believe that systematic documentation and analysis of how the 
program has evolved would provide a valuable resource for program staff and the 
research community in general. This kind of evaluation would involve the use of a 
standardized protocol and survey that would investigate issues of adaptation and 
implementation. 


Establishing realistic and meaningful criteria for success 

After visiting a number of countries and reviewing evaluation findings from 
additional countries, it seems that it would be beneficial to start a conversation among 
program staff about how to define overall success, criteria for success, and specific 
indicators of success in ways that allow for meaningful and realistic evaluation of the 
program. As we discovered in the U.S. evaluation, much of the real impact of the 
program is in its ability to facilitate small but important developmental steps toward 
technology integration and to promote reflection on technology use among large numbers 
of participants, rather than radical transformations. When both proximal (near) and distal 
(far) indicators of success are identified, evaluations can be designed so that subtle and 
gradual transitions can be captured. 

The examples of teacher and student work presented to us during our site visits and 
the projects granted awards and space on the Intel Teach to the Future web site are 
impressive. Intel Teach to the Future staff should certainly use their evaluations to 
understand the conditions that need to be put in place to support this kind of work. 
However, exemplary projects are not the only manifestations of program success, and 
may not even be the best way to illustrate the true breadth of the accomplishments of 
Intel Teach to the Future. Highly informative evaluations can be designed that explore the 
gradual process of change that may not always involve the most striking achievements, 
but can shed light on the range of positive experiences Intel Teach to the Future trained 
teachers are having in their schools and classrooms. 
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Country Evaluation Summaries 


Over the past nine months we have been in communication with most of the education 
managers responsible for implementing Intel Teach to the Future. As we noted above, the 
various countries are at different stages of program implementation and evaluation. 
Below we provide brief summaries describing the evaluations plans, materials, and, in 
some cases, findings for the twenty countries from which we have received information. 


Asia 


Pakistan 

AC Nielsen is conducting the Intel Teach to the Future evaluation in Pakistan. The 
evaluation design includes both quantitative and qualitative methods. The quantitative 
methods include two end-of-training surveys and the Core Survey. Pakistan has regularly 
submitted its end-of-training survey data to EDC each quarter. The Core Survey will be 
administered face-to-face to 330 Master Teachers. The qualitative methods involve a 
series of three focus groups with eight Master Teachers, which have been completed. 

The key findings from the focus groups in Pakistan reveal that many of these MTs 
had no prior experience with technology, and even those who did had not used it in their 
teaching. They responded positively toward their training. In particular, they liked the 
methodologies presented, the idea of using technology as an educational tool, the fact that 
the training was “hands-on” and practice-based, and the opportunity to collaborate with 
their peers. In particular, they noted that the ideas conveyed in the training could help 
with creating presentations for classroom teaching, for student assessment, for providing 
additional opportunities for weaker students to review the material, and for encouraging 
students to conduct independent research and share information. They also felt that the 
skills they learned would help them with various administrative and instructional tasks, 
such as researching and preparing materials for instruction and maintaining student 
records. Their major complaints about the course were related to time. They suggested 
that it would help if they could be released from their classes in order to participate in the 
trainings. They suggested that the training would be strengthened if it provided subject- 
specific information about integrating ICT. They also suggested the program would be 
strengthened if there were some instruction regarding the hardware as well as the 
software because teachers are often faced with hardware problems that they do not know 
how to resolve. The obstacles they cited included lack of technical resources, lack of time 
designated in the school day for ICT use, and lack of administrative support (particularly 
in government schools). When asked about the resources they would like made available 
on the Intel web site, they suggested subject-specific materials, lesson plans from other 
teachers, a directory of other teachers in the program, a technical support areas for 
addressing technical problems, links to useful research and information about new Intel 
workshops. 
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China 

The evaluation of China’s Intel Teach to the Future program has been conducted over 
the past year by the Ministry of Education. There are a number of components to the 
evaluation in China. First, the MOE administers both the end-of-training survey and Core 
Survey, with an additional 2 questions of specific interest to China’s program staff. China 
has just completed the Core Survey data collection, and will be submitting these data to 
EDC by the end of December. The second component of the China evaluation is more 
formative in nature. Program staff make visits to schools in various provinces and regions 
and use a standard protocol to document the program activities taking place. These visits 
also include interviews with the local RTA leader and forum discussions with MTs. The 
China Intel Teach to the Future evaluation also has principal and student surveys that 
gather feedback on the program from these groups, a self-evaluation form for MTs, and a 
“best unit plan” award program, which enables the program staff to collect examples of 
exemplary teacher work. The Core Survey administration and data collection were just 
completed by the MOE. These data are now being analyzed. 


India 

The evaluation of the Intel Teach to the Future program in India is being conducted 
by IMRB. So far the evaluation is comprised primarily of three different surveys. The 
first is the EDC end-of-training survey, which India administers regularly. The second 
survey is the EDC Core Survey plus many additional questions, some of which are open- 
ended, others of which ask for demographic data, and still others that probe teachers on 
aspects of their teaching environment. The third survey is one given to principals in 
participating schools. This gathers demographic information, information related to 
school infrastructure, teacher responses to the program, and principals’ reactions to the 
program. Like the teacher survey, this includes both open-ended and closed questions. 

The findings from the principal survey indicate that there is a high level of support for 
Intel Teach to the Future. Principals give high ratings to the quality of the program, and 
report that ICT usage has increased in their schools, although this increase is higher in 
urban regions than rural ones. Principals feel that the program is relevant to and can help 
improve the quality of education. Again, these responses are higher among urban 
principals. The challenges they identified included problems with technology access and 
time to implement the new projects. Access to the Internet is particularly low in rural 
areas, with only 26% of rural teachers able to access the Internet, and 19% of students. 

The findings from the teacher survey indicate that there is a high level of satisfaction 
with the program, and that teachers’ use of ICT has increased. In addition, teachers have 
very positive responses to the pedagogical as well as the ICT elements of the program, 
particularly collaboration among students, students presenting their work to their peers 
and students conducting independent research with the use of ICT. The survey indicated 
that the teaching strategies presented in the training were both new and relevant to large 
majorities of the teachers (from 85%-95%). Nearly all teachers felt that student were 
more motivated and engaged when using ICT in the classroom (96% in urban areas and 
98% in rural areas). The problems identified with successful implementation by all 
teachers included lack of time for implementation, students’ poor computer skills and the 
fact that all of the materials were in English. In rural areas, teachers also cited lack of 
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access to technology as a significant obstacle. Teachers in both urban and rural areas 
indicated that they received support from their principals in the use of ICT. 


Philippines 

SEAMEO INNOTECH is conducting the evaluation in the Philippines. This 
evaluation consists of the Core Survey with additional questions, including open-ended 
questions. Most of the respondents to this survey were MTs. We will be receiving the raw 
data from this survey soon, and will integrate it with the data from the other countries. 

The findings from this survey indicate that MTs rated their computer skills more 
highly after they completed the training and that their frequency of computer use 
increased after the training. The survey data also reveal that roughly a third of the 
respondents made use of their manuals and CDs more than 10 times since the training. 
Fifty-six percent of the teachers said that they implemented technology lessons with their 
students at least once a month. Large percentages were highly positive about students 
responses to their ICT projects: 86% said that the students were motivated and engaged in 
the work; 86% said that the students helped each other; and 80% said that student work 
was more creative when they participated in their new technology-integrated projects. 
Most of the respondents (75%) felt that the teaching methods presented in their training 
were relevant to their teaching goals. They also reported using many of these teaching 
strategies in their teaching, such as using essential questions to structure lessons (68%), 
having students work in groups (64%) and allowing students to conduct their own 
research (61%). When asked how they would improve upon the program, respondents 
mentioned finding time within the school day for the training, providing follow-up 
training, and providing teachers with examples of work creating by fellow Intel Teach to 
the Future teachers. 


Australia 

We have a proposal from the Consulting and Development Unit of the Faculty of 
Education at Deakin University to conduct the evaluation of Intel Teach to the Future in 
Australia. The proposed work will focus on the states of New South Wales and Victoria 
and will be both formative and summative. Deakin proposes seven stages of work. 


Stage 1- Information Gathering: Clarification and information gathering; and 
collection of MT baseline data. 


Stage 2 — Senior Trainer/Master Trainer Program: Observation of the ST/MT 
program; information gathering from the senior trainers; and administration of the post 
training MT survey. 


Stage 3 - Master Trainer/Participant Teacher Training: Observation of selected 
training (four case studies); administration of post training PT survey; interviews with 
MT (4 case studies — Victoria and 3 case studies - NSW); classroom observation/student 
discussions; interviews with principals (4 case studies — Victoria and 3 case studies - 
NSW); interviews with regional/district personnel (4 case studies — Victoria and 3 case 
studies — NSW); and ongoing communication (case studies). 
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Stage 4 — Final MT Core Survey. 

Stage 5 - Attendance at MT Workshop (Victoria and NSW) 

Stage 6 - Final PT Core Survey 

Stage 7 — Final Report 

The proposal also includes detailed plans for sampling, data analysis, and reporting. 


Thailand 

Thailand’s program and evaluation are in their infancies. The Thailand Education 
Development Alliance (TEDA) has submitted a proposal to conduct the evaluation of 
Intel Teach to the Future, focusing on four major activities: (1) data collection and 
analysis of schools status; (2) observation of the ST-MT and MT-PT training to collect 
data and better understand the curriculum; (3) end-of half-year evaluation; and (4) 
compilation, data analysis and recommendations. Grounded in current educational theory 
and the status of schools in Thailand, TEDA is proposing a formative evaluation that will 
provide constructive feedback to the program for better integration into the schools of 
Thailand. 

The proposal seems well grounded in current theory and knowledge of the 
educational system. TEDA distinguishes between traditional and reformed educational 
practices. They discuss challenges to using technology, current practices, and how Intel 
Teach to the Future is likely to contribute. 

TEDA plans to begin with analysis and data collection of school status, initial site 
visits, and a study of the curriculum. They then will observe MT-PT training, followed 
by the end-of-half-year evaluation. They have included a detailed sampling plan as well 
as a data analysis plan for both qualitative and quantitative data. 


South Korea 

Researchers from the Chunchon National University of Education are conducting the 
evaluation in South Korea. This year the evaluation has focused primarily on the 
administration of the Core Survey, which has nine additional questions (including an 
open-ended question) and 4 additional options within questions from the Core Survey. 
The administration of the survey will be web-based. We expect to receive Core Survey 
data from South Korea at the end of December. The South Korea evaluation also utilizes 
the end-of-training survey, and we regularly receive these data from South Korea. 


Japan 
Nikkei Research is conducting the evaluation of Intel Teach to the Future in Japan. 
The evaluation currently consists of the Core Survey and the end-of-training survey. We 
regularly receive end-of-training data from Japan, and we recently received the Core 
Survey data from Japan, so we were able to include these in our analysis reported here. 
The Core Survey data show that about one quarter of the respondents report 
implementing a new technology lesson more than once a month, 15% have done so about 
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once a month, 26% implement less than once a month; and about one third have not 
implemented a new technology lesson. Ninety-nine percent of teachers report having 
access to a computer lab, and 99% of these have Internet access. However, the vast 
majority (84%) do not have classroom computers. There were few teachers who said it 
was “not true at all” that the teaching strategies presented in the training were relevant to 
their teaching goals. However, only 21% said that this was “very true.” The obstacles 
more teachers who did not implement identified as the reasons for this were related to 
time constraints rather than infrastructure issues. However, interestingly, lack of access to 
technology was one of the most commonly cited obstacles among those who had 
implemented. 


Taiwan 

Researchers from the National Taiwan Normal University are conducting the 
evaluation in Taiwan. This year the evaluation consisted of the end-of-training survey 
and the Core Survey. We receive quarterly end-of-training data from Taiwan. In addition, 
we received the Core Survey data from Taiwan and were able to use those in the analysis 
here in this report. 

The Taiwan Core Survey includes 12 additional questions. The findings from these 
additional questions indicate that nearly all of the respondents (94%) report that the 
reason that chose to participate in the training was to learn how to integrate technology 
into their teaching. Most teachers (90%) have access to computer labs and nearly all of 
these (99%) have Internet access. However, half of the teachers have no classroom 
computers, and 43% have only one computer in their classroom. A little over half of the 
teachers stated that they implemented part of their unit plans; 38% said they did not 
implement them at all. Twenty-eight percent stated that they did not implement any new 
technology project. 

The items most teachers agreed were obstacles to implementation included lack of 
access to hardware and software, time constraints and lack of administrative and 
technical/instructional support. Most of the teachers who responded (81%) to the survey 
do not work in a school with a MT. Over half of the respondents reported that they 
discuss technology with their peers more often since the training, and they assist their 
peers more often with technology integration. 


EMEA 


South Africa 

Neil Butcher and Associates are conducting the evaluation of Intel Teach to the 
Future in South Africa. The South Africa evaluation is happening in two stages. The first 
stage of the evaluation, which focuses on implementation, has already taken place, and 
the report was submitted. This report presented findings from initial site visits to 8 
schools in 3 provinces. The 2004 evaluation plan calls for a second visit to each of these 
schools, as well as regular phone interviews with the facilitators (Master Teachers) to 
receive updates on the program implementation. The evaluation also includes 
administration of the end-of-training survey and Core Surveys, which will begin in 2004. 
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Findings from the initial case studies show, first of all, the diversity of school 
environments in which the program is being implanted—from a group of rural schools 
whose only access to computers is at a community center to a well-resourced private 
urban school. Because this initial visit was conducted before teachers had been able to 
participate very much in the training, it reflected the expectations that teachers had of the 
program, and their apprehensions. Many felt that this training was providing an essential 
service to their teaching communities, but they were concerned that their skills and the 
resources available to them would not be adequate for them to take full advantage of the 
training. Even teachers in the well-resourced school, who noted that they have quite a 
large number of computers expressed concern that they rarely have the time to use them. 
Of particular concern to both participant teachers and facilitators was the lack of 
incentives for participating in and delivering the training. It is very difficult for some of 
these teachers to attend trainings because of lack of transportation. Because the training is 
so time-intensive, many felt that without incentive participation would be limited and 
inconsistent even among teachers with the desire to learn ICT skills. A number of 
teachers also felt that the curriculum lacked enough locally relevant examples and was 
too oriented toward American teachers and the American education system and . Many 
stated that the materials provided by the program, such as the CD-ROM, were very 
useful. 


Germany 

The Intel Teach to the Future evaluation in Germany, conducted by researchers from 
Dillingham University, has been going on for a number of years. Evaluators working 
with program staff in Germany developed and administered their own survey for the past 
two years, and have reported their findings each year to EDC and Intel. These surveys 
provide information about the teachers participating in the program, their technology 
skills, their access to technology in their schools, and their reactions to the pedagogical 
and technological aspects of the training. They have recently translated both the Core 
Survey and the end-of-training survey into German and have submitted drafts of these to 
EDC. The evaluators will begin administering these in 2004. The German evaluation also 
includes an evaluation of its pilot online training program 

The findings from the survey conducted in Germany indicate that the teachers 
participating in the training have a fair amount of experience with technology, and nearly 
all of them have home computers. Most do not have classroom computers, but they do 
have access to computer labs. More than half of the respondents reported that although 
the education system did offer ICT training in their areas, it was not enough, and that they 
did not feel these trainings were tailored to provide them with appropriate pedagogical 
support for implementing ICT in their teaching. A large majority had favorable opinions 
of the quality of their Intel Teach to the Future Master Teacher and the overall Intel 
Teach to the Future curriculum. Ninety percent of teachers said that they would enroll in 
the course again. Teachers reported higher skill levels after the training. A large 
percentage (89%) felt that the training would encourage them to integrate technology 
more in their teaching, and 70% claimed that they were motivated to change their 
teaching practice after the training was completed. 
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Russia 

The Russian evaluation is very comprehensive, covering many aspects of the 
program. The evaluation team comes from an organization called RELARN, a group of 
educational specialists. They have conducted their own surveys. Participant teachers and 
Master Tutors are given surveys at the beginning and end of their training. The evaluators 
in charge of administering the surveys also attend training sessions and interview Master 
Tutors and Participant Teachers. One of the evaluators also uses electronic 
communication with participants to collect the stories about teachers’ experiences 
implementing their new materials. There is also a self-examination survey for staff who 
run the various training centers. The evaluation group is also conducting a case study of 
the program in the town of Toliatti, which is the first town to establish a regional training 
center. The Russian Intel Teach to the Future program initially relied on the use of only a 
few training centers in major urban areas, which made it difficult for teachers in more 
remote locations to attend. This case study will shed light on the process of establishing 
regional Intel Teach to the Future training centers, and will provide information that 
program staff can use to establish future centers. The Russian evaluation also includes 
analysis of the unit plans that teachers develop in order to assess the quality of the 
training. The evaluators recently translated the end-of-training survey and the Core 
Survey into Russian, and will begin administering those in 2004. What makes this 
evaluation design particularly useful is that the evaluators are working in partnership with 
program staff, and maintain ongoing communication with them. Evaluation findings are 
used in a formative way, helping staff make decisions in order to improve the program. 

The findings from the Russian survey show that teachers have very positive responses 
to the training overall. They rate the quality of their Master Tutors highly, and also highly 
rate the opportunity to collaborate with peers. The survey indicates that before the 
training, only 29% of teachers integrated IT into their teaching, but after the training 70% 
reported doing so. Eighty-five percent were encouraging their students to use ICT in their 
work after the training, and 70% reported that they consulted with their colleagues on the 
use of ICT after the training. Most teachers (64%) reported using project-based learning 
in their classrooms after participating in Intel Teach to the Future. The main complaints 
about the program were that the training was not conducted locally (this problem is now 
being addressed by program staff) and that the trainings are comprised of teachers who 
have very different levels of ICT skills. 


Ireland 

Researchers from the University of Limerick and researchers from University College 
Cork conducted the evaluation in Ireland. We have received a report from the University 
of Limerick, and are expecting the report from University College Cork very soon. 

The Ireland program administers an end-of-training survey, which contains the 
international questions as well as additional questions about participant demographics, 
teacher practice and obstacles to technology use. This end-of-training survey is very 
much like the one used in the U.S. We regularly receive end-of-training data from 
Ireland. This program also has a teacher follow-up survey, very much like the one that 
has been used in the U.S. The follow-up survey has many questions similar to the Core 
Survey, but it is not the Core Survey, and, therefore, the data can not be aggregated with 
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the international data. The Ireland program began administering its follow-up survey 
before the Core Survey was created, so they were not able to adapt this. The Ireland 
evaluation also includes interviews with Participant Teachers, Master Tutors, IT advisors 
and two Participant Teachers who did not complete the course. The evaluators are also 
conducting observations of the training. 

The findings from the evaluation indicate that teachers have a very positive reaction 
to their training; 95% say that the information they gained through the training would 
help them integrate ICT in their classrooms, and 96% would recommend the training to a 
friend or colleague. Only 38% stated in the follow-up survey that they had implemented 
their unit plan. A quarter (26%) of those who had not done their unit plans in their 
classrooms stated that they had created the unit plan too late in the year to implement it, 
and 17% that they did not have adequate technology available. Other obstacles to 
integration included lack of time in the curriculum, lack of technical support and lack of 
Internet access. In the follow-up survey, teachers continued to have a positive response to 
the training. Seventy percent stated that the Intel Teach to the Future training had been 
“very useful” and 27% said it was “somewhat useful.” More than 80% of participants felt 
that the CD-ROM resource was useful, and 87% of tutors and participants felt that the 
manual was useful. 


United Kingdom 

A final report from the U.K. was issued in July of 2003, and authored by Penni Tearle 
and Patrick Dillon of the University of Exeter. This comprehensive report highlights the 
three major research activities that Intel Teach to the Future undertook. The first activity 
was a critical review of the printed materials used in the program. The second activity 
was the administration of a survey distributed to teachers, “Key Trainers,” and tutors. The 
final research activity was a set of case studies of various institutions (schools, higher 
education institutions, and school centered teacher training) that provided initial teacher 
training. 

Findings from the research indicate that Intel Teach to the Future was generally well 
received and that it meets a range of needs for teachers as well as student teachers. It was 
found that the program addresses a need for the development of basic ICT skills, but that 
the level of the program was targeted for individuals with limited ICT knowledge, skills, 
and experience. 

The research further found that the flexibility of the program was an important 
characteristic. Most key trainers and institutions, in fact, adapt the program to particular 
needs. 

In terms of specific programmatic feedback, the research indicated that the paper 
materials were clear and easy to follow. Most respondents felt that the program was 
relevant to their teaching, but it lacks subject-specific materials. Positive characteristics 
include the face-to-face interactions, small group size, and the work at your own pace 
structure. The program helped to increase teachers’ confidence. The major piece of 
negative feedback was that the program was very time consuming. Teachers were 
reluctant to devote after school hours or weekends to such training. Simply, the out of 
school hours were too difficult and an undesirable characteristic. There also was a 
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concern that Intel Teach to the Future was one among many other programs available that 
address the need to develop ICT skills. 


Israel 

The evaluation report from Israel was written in July of 2002 by researchers from the 
Achva College of Education. The methods used were two questionnaires, one asking 
teachers about their expectations for the program, and another asking them to assess the 
program. A first finding is that MT’s gave the program higher evaluations than did PT’s. 
Findings indicated that there were two main reasons for participating in the program. A 
first reason was to learn effective integration of technology into educational practices. 
The second reason was for personal reinforcement and empowerment. The main benefits 
for participating in the program were to expand teacher knowledge of technological tools 
for instruction, gain exposure to innovation, satisfy curiosity, and acquire instructional 
tools. 

The Israeli evaluation attempted to measure the degree to which the program was 
implemented and rated its implementation as medium to high. Issues such as teacher 
knowledge, infrastructure, and support were noted as factors in the degree to which the 
program was implemented. 

Satisfaction with the program was reported as high for both MT’s and PT’s. Intel 
Teach to the Future seemingly added value to teachers’ knowledge of tools, increased 
their skills, and their ability to teach effectively. 


Italy 
The program in Italy has started administering the end-of-training survey. We 
received end-of-training data from Italy for Q3-04. 


Latin America 


Costa Rica 

Two independent evaluators (Sylvia Nufiez and Wendy Jimenez) submitted an 
evaluation report for Intel Teach to the Future in Costa Rica in July of 2003. The 
evaluation identified the following significant achievements of the Technical Assistants: 
(1) increase in abilities and technological skills; (2) more interest in the computer field; 
(3) clarification of vocational aspects in the short term (career choice), mostly in the 
computer area; (4) improvement in social skills (expression of ideas, explanation of 
concepts, interaction with others and public speaking); (5) increase in self-esteem, more 
self-confidence in themselves and their skills (decrease in anxiety, fear and nervousness); 
(6) social, community, and family recognition; and (7) development of values and 
attitudes such as patience, tolerance, perseverance, respect, and solidarity. 

The most important limitations of the program were found to be: (1) little time to do 
training activities; (2) inadequate infrastructure and poor condition of computer resources 
and Internet connections; (3) very large groups; and (4) handing in licenses to just one 
teacher per group. 

The evaluators proposed the following solutions: (1) extend and redistribute time in 
the activities during the modules; (2) improve the infrastructure and condition of 


EDC/CCT Intel Teach International Interim Report 
96 Morton St. NY, NY 10014 Page-36 


computer resources — mostly Internet; (3) use previous diagnostic of knowledge for group 
leveling; and (4) make smaller groups with more flexible schedules. 

Additional findings included improvements in pedagogy and an understanding of 
constructivism. It was also found that teachers received social and institutional 
recognition based on their association with the program and Intel. 


Brazil 

Intel Teach to the Future was introduced in Brazil in June of 2001. The goal for 2003 
was to train 20,000 teachers in seven states. We have received a document from the 
Carlos Chagas Foundation that reports on the pilot assessment of the program in Brazil. 
Initial results indicate that the program has impacted teacher values and beliefs about the 
use of technology in education, as well as their ability to integrate technology into their 
teaching activities. The program provides an opportunity to acquire new knowledge of 
technology and teaching tools. However, the amount of change in pedagogical practice is 
affected by four factors: (1) access; (2) infrastructure; (3) support by the school; and (4) 
teacher’s prior knowledge of technology. It was found that teachers who use Intel Teach 
to the Future have better technological infrastructure in their schools, go to the lab more 
often, have higher levels of education, and had more prior knowledge of technology 
before the training. Differences were found between public and private schools. Private 
schools provide more administrative support for the program than do public schools. 
Income apparently is an intervening variable between types of schools. 

Most respondents believe that Intel Teach to the Future is useful, but there are still 
difficulties in using the technology. Infrastructure seems to be a major issue. More than 
half of the respondents have no computers in their classrooms, and more than one-quarter 
have no Internet access in their schools. These facts limit the extent to which the 
program can be implemented. Further, there was not enough time to sufficiently cover the 
amount of materials in the program. There simply was too much information. A further 
finding about the course materials was that respondents thought they were easy to access 
and understandable. However, this was influenced by the teacher’s level of prior 
knowledge of technology. Teachers with prior knowledge found the materials easier to 
use, than those without prior experience. Respondents also reported that they found the 
pedagogical theory somewhat new. 


Mexico 

Concierto, an independent research firm that has previously conducted an extensive 
evaluation for the MOE in Mexico, is conducting the evaluation in Mexico. The 
evaluation involves a review of state and federal documents (which include web-based as 
well as paper documents) related to ICT planning and infrastructure in Mexico, an 
administrator survey, site visits to schools participating in the program, interviews with 
participants and the administration of the end-of-training survey and Core Survey. The 
program in Mexico intends to administer the surveys online, and is currently developing 
the technical infrastructure to enable web-based survey administration. 
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Argentina 
Currently, the evaluation in Argentina consists of the end-of-training survey and the 
Core Survey, which has been translated into Spanish. We regularly receive end-of- 


training survey data each quarter from Argentina. No additional questions were added to 
the Core Survey for Argentina. 
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Conclusion 


Evaluation findings to date suggest that Intel Teach to the Future is a valuable 
professional development program for teachers working in a broad range of contexts. The 
program provides ongoing and continuous support for the development of teacher skills 
and knowledge in project-based learning and general ICT literacy, which initial 
evaluation findings indicate is crucial to its success. Intel Teach to the Future has learned 
from the limitations of past programs that provided substantial hardware and software to 
educational institutions but gave little or no thought to how to support the long-term use 
of those resources. Instead, its personnel have invested enormous energy in nurturing 
long-term and sustained relationships with governmental and educational stakeholders in 
order to allow the program to take hold and flourish. 

There is considerable variation among the countries represented in this report 
regarding the maturity of their Intel Teach to the Future programs, and in the structure 
and design of their programs. EDC and Intel Teach to the Future program staff need to 
ensure that evaluation is used to help all of us understand what success can look like in a 
variety of cases. Because of the unique constraints, challenges and opportunities 
experienced by participants in each country, program effectiveness must be defined and 
described locally, with specific contextual factors taken into consideration. At the same 
time, program-wide definitions of success need to be further articulated so that program 
staff and evaluators in countries around the world can create evaluation designs that have 
at their foundation evaluation questions derived from consistent large-scale goals. 

Despite the substantial differences across sites, many commonalities exist. 
Particularly striking is the pervasiveness and continuity of the factors that facilitate or 
hinder technology implementation. The same four factors—infrastructure, professional 
development, administrative support, and time—emerge time and time again as the 
variables within schools as systems and learning organizations that make a difference in 
teachers’ ability to integrate technology into their teaching. 

Future work and, perhaps, targeted case students will focus on these factors and 
emerging themes to help tease out the most influential variables and the interactions 
among them that enable programs like Teach to the Future to meet objectives and attain 
global impact. Those objectives may range from one student in a rural community in 
India, Australia, or Russia preparing an in-depth report on a particular topic to a 
pedagogical transformation of not just a classroom but an entire school in China. 
Effectiveness must be measured in proximal, rather than distal steps. Intel Teach to the 
Future has laid crucial groundwork to allow each country’s program to be evaluated on 
its own terms, and our understanding of these evaluations to date suggests that, over time, 
we will be able to document and describe how each site moves ahead. In doing so, we 
look forward to helping the Intel Teach to the Future program create a comprehensive 
knowledge base that can serve as a resource for program staff and participants 
worldwide. 
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